Skip to content

fix: Prevent context attributes from influencing judge template parsing#129

Open
jsonbailey wants to merge 4 commits intomainfrom
devin/1775752761-fix-judge-template-injection
Open

fix: Prevent context attributes from influencing judge template parsing#129
jsonbailey wants to merge 4 commits intomainfrom
devin/1775752761-fix-judge-template-injection

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented Apr 9, 2026

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Addresses SEC-8020. Mirrors the fix applied in the Go server AI SDK (go-server-sdk commit 3317871). A parallel PR has also been opened for launchdarkly/js-core.

Describe the solution you've provided

The judge's _interpolate_message method previously used chevron.render() (Mustache templating) for its second-pass interpolation of {{message_history}} and {{response_to_evaluate}} placeholders. This is vulnerable to template injection: attacker-controlled values resolved during pass 1 (e.g. Mustache delimiter-change tags like {{=[ ]=}}) would be interpreted as control syntax by the second Mustache pass, potentially blinding the judge to the actual content being evaluated.

This PR replaces the Mustache-based interpolation with simple str.replace() calls. Since the judge only ever substitutes two known placeholder strings, a full template engine is unnecessary, and literal string replacement is both safer and simpler.

Note: chevron is still used in client.py for the first-pass template interpolation (which is expected behavior — pass 1 needs Mustache to resolve context variables).

Describe alternatives you've considered

  • Escaping attacker-controlled values before passing them to Mustache for pass 2. This is fragile and hard to get right across all Mustache control sequences.
  • Using a different template engine with sandboxing. Unnecessary complexity given only two fixed placeholders.

Additional context

Key items for review:

  • Verify that the chevron import removal from the judge module doesn't affect other code paths (it doesn't — chevron is only used in client.py for pass 1).
  • The regression tests cover all major Mustache injection vectors (delimiter changes, partials, comments, triple-stache, sections) plus edge cases like Mustache-like syntax appearing in the actual message content.

Updates since last revision

  • Simplified the _interpolate_message docstring per reviewer feedback on the parallel JS PR — now a concise two-line summary instead of a verbose multi-line explanation.

Link to Devin session: https://app.devin.ai/sessions/651e799b906748a4834bafefb4a3e3e5
Requested by: @jsonbailey


Note

Medium Risk
Changes judge prompt interpolation logic (and removes Mustache rendering) to mitigate a template-injection vector; could subtly alter how existing judge message templates with non-{{word}} placeholders are rendered.

Overview
Prevents template-injection in judge prompt construction by replacing chevron.render in Judge._interpolate_message with a regex-based substitution that only expands {{message_history}} and {{response_to_evaluate}}, treating other Mustache control syntax as literal text.

Adds targeted regression tests covering delimiter changes, partials/comments/sections, repeated placeholders, and cross-placeholder injection to ensure attacker-controlled values cannot affect the second-pass interpolation.

Reviewed by Cursor Bugbot for commit e64c5d2. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration bot added the devin-pr Pull request created by Devin AI label Apr 9, 2026
Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
@jsonbailey jsonbailey marked this pull request as ready for review April 13, 2026 21:50
@jsonbailey jsonbailey requested a review from a team as a code owner April 13, 2026 21:50
… injection

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1ad110e. Configure here.

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devin-pr Pull request created by Devin AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant